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Whole-School Bilingual Education Programs: 
Approaches for Sound Assessment 



What are whole-school bilingual education programs? 
How do we know when they are functioning effectively for all students? 

Although we understand bilingual education to mean dual language instruction (and know that dual 
language instruction has been available in the US since before colonial times)> few, if any, whole schools 
have operated as academies of bilingual education since World War I (Lessow-Hurley 1990). However, 
under the 1994 reauthorization of the Elementary and Secondary Education Act (ESEA), specifically under 
Title V1I> schools are able to develop and implement exemplary whole-school bilingual education pro- 
grams. This funding is tied directly to the systematic improvement and reformation of education programs 
serving language minority (LM) and limited English proficient (LEP) students. 

For educators involved in the instruction of LM and LEP students, the challenge is to identify and 
determine those features that comprise effective bilingual schools. To meet this challenge, it is necessary to 
rely on knowledge from the field or bilingual education and the studies on instruction and program 
effectiveness. Together, these features hold great potential for explaining how a whole-school bilingual 
education program can function. 

The literature is replete with studies that identify key indicators useful for designing sound whole 
school bilingual education programs (c.f, Lucas, Donato, and Henze 1993; Ramirez, Yuen, and Ramey 
1991; and Tikunoff 1985, 1991). We were able categorize these indicators into three areas: 

1. School context indicators describe the ethos, management, and resources that permeate and influence 
the attitudes of school staff, students, and parents in language minority communities; 

2. School implementation indicators target key features in bilingual education schools including curricu- 
lum and instruction, staff development, the responsibilities of administrators, and the role of parents; 
and 

3. Student outcome indicators identify the skills and strategies required of limited English proficient 
students to succeed in whole-school bilingual education programs and to attain the performance 
standards outlined in Goals 2000. 

Although the features in each category are similar to those in effective mainstream schools (Bossert 1986), 
we found distinct cultural and language- related characteristics. These indicator are presented in Figure 1 
along with brief descriptions of their supporting features. 

It is with these effective school features in mind that we suggest assessment strategies that can be 
incorporated into the design and measurement of effective whole-school bilingual education programs. 
However, several qualifications must be presented before attempting to apply the identified features to 
programs for LM students. From an optimistic perspective, these features: 

1. offer schools with diverse student populations the possibility of becoming not only equitable and 
desirable places of learning for all students, but also learning environments for the adults and 
community members who work or volunteer there; 
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2. expand upon the characteristics identified in studies of effective schools and instruction. Features such 
as family involvement, bilingualism for all students, cultural and ethnic sensitivity, and cross-cultural 
communication are elements not identified until recently as critical to schools with large LM popula- 
tions; 

3. validate practices that have been employed by bilingual educators for several years and arc believed to 
lead to more successful students and programs; and 

4. lend themselves to assessment and to meeting the evaluation regulations for programs funded under 
the Elementary and Secondary Education Act (ESEA). 

However, we also must be cautious in recognizing potential limitations with these features. They: 

1 . should be viewed as necessary, but not sufficient, elements for school improvement. These features are 
derived directly from the literature on effective instruction and programs for language minority 
students. Other features may also need to be examined in shaping whole-school bilingual education 
programs. In addition, further research is needed to understand the relationship between effective 
school features for LM students and academic achievement; 

.2. have not been established as having a cause-and-effect relationship with student achievement. How- 
ever, these features have been found in schools that have above-expected levels of student achievement; 

3. may not be transferable to other schools. Nonetheless, these features do provide some sound guidelines 
for ensuring an equitable and appropriate education for LM students; and 

4. arc drawn from a small number of studies. Some features may be more important for particular 
programs or types of programs than others. 

PURPOSE 

The primary purpose of this document is to suggest (1) how the features of effective bilingual 
education programs can be applied to whole-school bilingual education programs and (2) methods for 
assessing whether these features are working. Unfortunately, some of the terms of assessment have technical 
meanings that can cause confusion. To ensure a common understanding of these terms, Figure 2 provides 
a glossary of terms used in this document to help the reader understand their precise meanings. Once this 
common undcrstan ling is ensured, we briefly describe what the literature tells us about each indicator and 
its key features. We then introduce a variety of sample assessment methods that can be used to measure 
these features as well as information on how to use these assessment approaches within a comprehensive 
evaluation plan. We conclude this document by listing several implications for conducting a sound 
evaluation of whole-school bilingual education programs. 

One caveat must be stated: we need to recognize that integrating effective features into a whole-school 
bilingual education program has rarely been attempted in the last 20 years. The system is complex and, 
therefore, somewhat uncertain. It will take some time to develop a whole-school bilingual education 
program, and more time to develop a sound assessment system. Immediate positive effects should not be 
anticipated. It will take administrators, staff, students, and communities time to "buy in" to the concept of 
a whole-school bilingual education program. However, the benefits for the community-at-large are great 
and encourage us to proceed. 
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Figure 2 

Glossary of Assessments Used to Measure Key Indicators 



Observation checklists. Checklists can be used to determine whether particular behavioral, physical, 
or environmental characteristics are present. Typically, desirable behaviors are described briefly and an 
observer checks (l/) whether each behavior is observed during a particular period of time (e.g., the 
first week of school). Scores can be developed by counting the number of checks. When the same 
checklist is used periodically throughout the program, it can be used to demonstrate progress. 

Surveys. Surveys usually list a series of questions to be answered orally or in writing by the respondent. 
The responses can be forced choice, where the answers are provided (e.g., Are you pleased with the 
language abilities of the staff? Yes/No), or Likert-scale continuum (4 to 7 response options such as 
'almost always" to "almost never"). The responses can also be open-ended, where the individual 
provides an answer (e.g., What pleases you most about the language abilities of the staff?) Scores can 
be developed by assigning point values to the responses (e.g., Yes=l, No=2) and summing these 
values. 

Interviews or fonts groups . Both interviews and focus groups can provide in-depth information. In. a 
structured interview, responses to a set of prepared questions can be recorded by the interviewer who 
can ask clarifying questions. Focus groups can include small groups of individuals and a facilitator to 
discuss a specific topic. Generally, scores are not developed. 

Standardized tests . Standardized tests are used to measure student skills. They arc so named because 
their administration, format, content, language, and scoring procedures are the same for all stu- 
dents — they have been "standardized." Locally developed and commercial standardized tests are 
available for most achievement areas and for some aspects of language proficiency. 

When referring to standardized tests, most people think of norm-referenced tests (NRTs); within 
this document, we refer to commercially available language proficiency and achievement tests as 
NRTs. NRTs are typically used to sort people into groups based on their assumed skills in a particular 
area; they arc useful when selecting students for participation in a particular program because they are 
designed to differentiate among students. In addition, NRTs can provide general information that 
will help to match classrooms for overall achievement levels before assigning them to a particular 
program. 

Alternative assessments. Alternative assessments are types of measures that fit a contextualizcd 
measurement approach — they "can easily be incorporated into classroom routines and learning 
activities. ..Their results are indicative of the student's performance on the skill or subject of interest" 
(Navarrete, Wilde, Nelson, Martinez, and Hargett 1990, p. 2). As used within this document, 
"alternative assessment" subsumes authentic assessment, performance-based assessment, informal 
assessment, ecological assessment, curriculum -based measurement, and other similar forms that 
actively involve the learner. 

For many types of alternative assessments, different types of scoring methods can be used. Three 
typically used methods are holistic scoring, which provides a general, overall score, primary trait 
scoring, which defines particular features (or traits) of student performance and then provides 
separate scores for each trait, and analytic scoring, which assigns a weight based on the importance of 
each trait (e.g., spelling might be weighted less than grammar). Identifying where and how errors 
occur allows teachers to determine where students understanding of a topic begins to deteriorate. 
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SCHOOL CONTEXT 

The context in which a whole-school program operates (whether or not it is a bilingual program) is 
essential to the universal well-being of students, teachers, families, and, ultimately, the program. According 
to recent research, the most important school context factors for students, especially LM students, are the 
overall culture of the school, its assets, and the administrations views toward the school and the program 
(see Figure 1 ). This section briefly describes these program context factors as they relate to diverse language 
populations and suggests ways to measure the factors to inform program improvement efforts. 

Ethos 

Ethos in a whole-school environment refers to the expectations, social norms, and contributions of 
staff, students, and administrators that enable all those in a school to function successfully in the learning 
environment. Think about a visit to an effective school. The attitudes and behaviors of students and 
teaching staff give an observer the feeling of pride and accomplishment. This "feeling' is the culture of the 
school, the ethos. 

Based on studies of effective schools for LM students, ethos can be broken into three general attributes: 
(1) the climate of the bilingual program and the whole school, (2) the expectations that staff have for all 
students (including LEP and English proficient students), and (3) staff and student morale. Each of these 
attributes arc described in the following section. 

Climate . The degree to which languages and cultures arc respected and affirmed is a key element in the 
climate of the whole-school program. It is important that the school climate be equitable toward the 
various ethnic and racial groups, the languages spoken other than English, and both sexes. Climate affects 
not only attitudes toward teaching and learning, but also more general attitudes and behaviors toward other 
cultures, education, and even employment possibilities. Characteristics to look for within a positive whole 
school program include: 

the use of inclusive, positive language (language that is free of racial, ethnic, linguistic, or sexual slurs); 
expressed knowledge about the demographic groups in the school (correctly naming ethnic groups and 
their home countries, identifying events and people relevant to the groups, and so on); 
the embracing of all languages as valuable vehicles for instruction and assessment; 
the inclusion of students families in school events; and 

the encouragement of communication among staff and students regardless of status or language. 

Within such a climate, all students, regardless of background or experiences, can feel acceptance, have 
enhanced self-images, and develop a desire to learn. 

Expectations . All students need to know that high, yet realistic academic and behavior standards are 
expected of them. Though it is not unusual to find high expectations for fluent English proficient students, 
limited English speakers often receive the message that less is expected from them than from their peers. To 
ensure that high expectations arc developed and maintained for all students, whole-school programs 
should: 

provide high level and challenging classes in languages other than English; 

ensure that academic and behavioral expectations are concrete, developed with student input, and 
equitably enforced; 
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encourage active student participation and provide quick feedback on assignments; 
hire qualified language minority teachers and paraprofessionals; 

encourage assessment methods that can be summarized to show student growth and accomplishments 
and can be shared with students, parents, and teachers; 

include language minority students in English-only classrooms when Feasible; and 
celebrate all students* achievements in a public manner. 

When building a positive climate for a diverse community of students, it is not only important that high 
expectations be set, but that the staff clearly believe that all students can meet these expectations. 

School morale . School morale is a good indicator of a whole school bilingual programs success — all 
students see themselves as successful learners, teachers sec themselves as successful educators, and adminis- 
trators sec themselves as successful creators of a learning environment; all want to be in school and to work 
together. Key features of a high-morale, whole-school bilingual education program include: 

culturally and linguistically diverse student government and teaching staff; 
long-range and short-range plans developed by diverse students, staff, and parents; 
attendance at school events reflecting the diversity of students and staff; 

positive attitudes about the school expressed by students (and staff) indicating a desire to be in school 
(as opposed to indicating that they arc required to be in school); and 
a clean campus that reflects the positive attitudes of staff and students. 

Although it is difficult to develop a plan specifically to improve school morale, it is quite possible to 
determine that morale is in need of a "fix/ 1 If morale is a problem, it is best to look toward other elements 
within the whole-school programs life that might affect it. 

Assessment of ethos . The relevant features of program context should be assessed on a periodic basis. 
Some appropriate means include observation checklists, attitude surveys, interviews, and focus groups. 
These assessment approaches can be used together, in combinations, or separately depending on the needs 
of the program to demonstrate an effective school context. Figure 3 shows samples of these approaches for 
assessing the school context indicators. 

Management 

Within the context of a whole-school bilingual education program, "management" refers to the 
incorporation of a system of supervision into the school that ensures the integration of effective practices 
for all students across key elements of the school environment. Effective management depends on the 
development of successful strategics for integrating these practices into existing school structures and 
supervising or monitoring the implementation of desired practices. 

Integration . Instruction that is effective and relevant for all students should be integrated in the whole 
school curriculum, not just added as a unit here or there, or treated as a special program decoupled from the 
regular curriculum (e.g., a "pull out" or ESL program). Some dimensions that should be considered when 
integrating effective practices into a whole-school context are to: 

emphasize effective practices for both English learners and second language learners across all parts of 
the curriculum; 

select curriculum materials in all content areas that reflect multicultural perspectives; 
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emphasize the integration of effective practices for all students in content area instruction; 
focus staff training on increasing multicultural awareness and awareness of community needs; 
involve parents, students, and school staff in planning and restructuring efforts; 

provide equal access for all students to programs serving exceptional students by modifying referral 
processes to accommodate language minority students (e.g., use of alternative assessments in place of 
commercially available norm-referenced tests); and 

assess student progress, to the degree possible, by matching the content, format, and language of 
instruction. 

Monitoring/Supervision . Administrators of whole-school bilingual education programs must consis- 
tently supervise and monitor assessment, placement, and instructional practices to ensure equitable and 
effective services for all students. Features of sound supervision include informing, delegating, and working 
with staff in planning and carrying out school practices. Areas that might be monitored include the 
incorporation of language and culture, instructional practices, testing and assessment, materials selection 
and development, staff qualifications and professional development, parent and community involvement, 
and referral procedures. 

Assessment of management . Assessing how well a whole school is actually being managed can be 
facilitated through the use of questionnaires and surveys completed by staff, families, and/or students. Self 
reflections, surveys, and observation checklists focusing on specific management issues can also be used. 
These approaches should be designed both to provide constructive criticism and to document practices. 
Feedback based on monitoring should point out strengths and needs. Where needs (i.e., activities 
inconsistent with effective practice for language minority students) arc identified, administrators must be 
able to provide suggestions and support for making improvements. Inscrvice staff training, observations, 
materials acquisition, and staff collaboration are some support activities that can be managed to facilitate 
the implementation of effective whole school programs. It is important that schools tailor their assessment 
of management to meet their particular needs and purposes. Figure 3 presents sample items and formats 
that might be used. In addition, Lucas (1993) has developed a detailed checklist with management 
indicators for effective secondary schooling for language minority students. 

Resources 

Resources arc the tangible and not-so-tangiblc commodities that make a service like education possible. 
Wc usually think of resources in their most tangible forms such as the school budget or building facilities. 
Some of the not-so-tangiblc resources include a dedicated and inspired teaching staff, flexibility to 
brainstorm and implement new and creative solutions, and community support. Within the context of the 
program, resources can be divided into three areas: administration, capacity, and time. 

Administration . Resources should be equitably administered and distributed throughout the school to 
ensure that the physical environment is clean, well-lit, and positive for all students and staff Administra- 
tive resources can include specialized personnel such as a program coordinator and support personnel who 
can schedule meetings, find substitutes, recruit native language volunteers from the community, explore 
training options, screen applicants for staff positions, acquire materials such as native foods and spices, find 
print materials in native languages, procure arts and crafts supplies, arrange field trips, and disseminate 
information. 
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Figure 3 

Sample Assessments for School Context 



Sample Ethos Behavior Checklist 

Directions: Cheek (/ ) each behavior that is usually seen. 
Performance Statements September January 

1. The staff uses inclusive, positive language. J d 

2. Classroom expectations are clear and concrete. ZJ J 

3. The staff is ethnically and linguistically diverse. J\ CI 

4. Attendance at school events is high. -J LI 



May 

□ 
□ 
□ 
□ 



Sample Resources Self-Assessment Survey with Dichotomous Responses 

Directions: Circle the response that most closely fits what you do or see on campus. 

1 . Meetings were scheduled regularly for planning. Yes No 

2. Money was budgeted tor substitute teachers for inscrvicc training. Yes No 

3. I'imc was used to evaluate the use of new instructional materials. Yes No 

4. Time was available for meeting with colleagues. Yes No 



Sample Management Survey with Likert-Scale Responses 

Directions: Circle the response that you feel describes your program. The response scale is (I) most of the time . 
(2) frequently , (3) sometimes , and (4) rarely . 

1. Curriculum materials in science reflect multicultural perspectives. 12 3 4 

2. Inscrvicc training focuses on multicultural awareness. 12 3 4 

3. Training is available that focuses on effective practices. 12 3 4 

4. Support services and extracurricular activities are available. 12 3 4 



Sample Open-Ended Interview 

Directions: Ask the first question. If the person does not respond, or if fit rt her clarification is needed, consider 
the prompts that are italicized. If you ask further questions, be sure to write them down with the person 's 
response. 

1 . How well do teachers* curriculum materials reflect multicultural perspectives? Are these materials 
available in all content areas? If not, in which areas are they lacking? 

2. Have staff dcvelopmcnt/inservicc sessions focused on multicultural awareness? What are some 
examples? 

3. Are effective practices (e.g., total physical response, natural language approach) integrated into 
content area instruction? if not, in which areas arc they not integrated? 

4. Are classroom expectations clear and concrete? Do students know: how papers will be graded: what 
behaviors are acceptable in class: or what the consequences will be for misbehavior? Can you suggest 
ways to improve students 'understanding of the expectations? Do parents understand these expectations? 
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Capacity . Capacity can be defined as the expertise of a staff to create and use effective new ideas as well 
as to improve existing strategics to maintain and improve the education of language minority students. Key 
features of capacity in an effective whole-school bilingual education program include: 

a guiding mission statement or philosophy thu embraces diversity; 

the inclusion of parents, community members, students, support staff, and others involved in the 
program in planning and decision-making; 

the establishment of regular meetings for training, planning, brainstorming, and creating; and 
a practical application of the ideas generated in planning sessions. 

Time. Without a sufficient amount of this most precious and basic of resources it is very difficult to 
establish and maintain an excellent schoolwide program. Time is necessary for adequate planning, training, 
communication, networking and sharing strategies and ideas, assessment and evaluation, pilot-testing new 
ideas or methods, and evaluating the results of a new method or program, among others. When considering 
the cost of in-service training, for example, administrators must consider the budgetary and time con- 
straints of luring substitute teachers or paying teachers to attend training during their time-off. Time 
provides countless opportunities for systematic growth and improvement and is a critical feature of effective 
language minority programs. 

Assessment of resources . In whole-school program settings with substantial management input from 
parents, community members, teaching staff, and other key stakeholders, formal budget and records 
reviews can also be used to assess program context. Prior to a record or budget review, the evaluation 
committee (composed of key stakeholders and internal and external evaluators) should identify priorities 
and reach consensus on questions they wish to answer during the review process. Some examples of 
assessment instruments for identifying available resources are presented in Figure 3. 

School Context Summary 

Program context is the first essential component for a whole-school bilingual education program. No 
education program can flourish if the context in which it operates is not conducive to success. The key 
elements for a whole-school bilingual education program are the overall climate of the school, its 
management, and the equitable use of its resources. In identifying these three issues as the focus for a 
whole-school program, we must know whether they arc "working." Also indicated here have been methods 
for assessing whether these three elements of the school are truly inclusive, flexible, and democratic, and 
whether they meet the needs of all students and their families. In other words, does the program context 
benefit the school experience of all staff, students, and family members? Once the context is in place, a new 
program can be considered for implementation. 

PROGRAM IMPLEMENTATION 

Though the school context is critical to the success of an effective whole-school bilingual education 
program, the actual implementation of its activities and programs has a direct impact on student success. 
Implementation includes specific school operations within the areas of curriculum and instruction, staff 
development, the role of the administrator and family/community involvement. This section describes 
those key elements of program implementation and ways in which these elements might be assessed to 
measure the operation of a program with diverse student populations. 
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Curriculum and Instruction 

Traditionally, curriculum and instruction have been organized around subject areas or disciplines and 
presented as separate entities using facts and skills that arc disconnected, fragmented, and disjointed. The 
teacher has been viewed as knowledge disseminator and gatekeeper; paper-and-pencil tests have been the 
"true' measures of knowledge. With the increased awareness of the needs of students from diverse 
backgrounds, trends in school reform have emerged with a new vision. This new vision looks at authentic 
learning and assessment as involving interaction with the environment in such a way that these experiences 
become integrated in the students' system of meaning (Bcanc 1991). Thus, active learning based on 
challenging content is viewer 1 as an effective way to motivate students to reach high levels of achievement. 
The role of the teacher as facilitator and meaning-maker and the ongoing evaluation of student progress 
through performance-based assessments arc emphasized in effective programs serving LM students. 

Pivotal to the implementation of effective curricula and instruction arc cultural and ethnic sensitivity; 
flexibility within the curriculum, instruction, and assessment; and successful English and native language 
teaching strategies. In the area of curricula, these features can be observed through: 

a curriculum grounded in the languages, knowledge, and background of the students; 

themes and activities that emerge from actual questions and needs of the students rather than solely 

through prescribed curricula or presumed needs and characteristics; 

a variety of courses offered to students in their native languages and in English; 

an inclusion of all students in bilingual programs; 

the establishment of national, state, and local standards for all students; 

the use a variety of new assessment mechanisms for holding schools accountable for reaching high 
standards; 

the availability of extracurricular activities such as newspapers, enhancement programs, and sports 
activities for all students, including language minority students; 

the involvement of educators, students, communities, businesses, and special interest groups in 
planning, decision-making, setting goals, and instructional support; and 

appropriate identification procedures for students with exceptionalities; programs are developed to 
support their exceptional needs. 

Similarly, there are key characteristics of instruction that relate to sensitivity, flexibility, and appropriate 
teaching strategies. These include: 

opportunities for students to develop their native language while develop : ng English; 
instructional practices that integrate content areas, promote higher order .''inking skills, and provide 
opportunities for active student responsibility and purposeful use of language and other skills; 
effective teaching strategics such as cooperative learning, language experience, readers' and writers' 
workshops, and ongoing performance-based assessments; 

materials that are varied, dcvclopmentally appropriate, free of bias, up-to-date, accurate, and reflective 
of the language(s) and culture(s) of the students; 

expectations maintained by teachers, students, and families that all students can succeed at high levels; 
and 

teachers empowered to make instructional choices that honor their professionalism. 
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Although these key features may be conducive to effective curriculum and instruction, alone they 
cannot improve the quality of teaching and learning in schools serving LM students. A clear framework for 
change based on high standards, comprehensive planning, continuous professional development, flexibility 
to draw upon all resources, and clear accountability are also key factors in improving teaching and learning. 

Assessing curriculum and instruction . A sound assessment plan can be used to stimulate discussion and 
reflection. Any combination of the following may be assessed: effectiveness of implementation, appropri- 
ateness of course content, degree of student involvement, appropriateness of teacher performance, ad- 
equacy of classroom environment, extent of learning standards, or student outcomes. Assessments may 
also focus on identifying the needs and characteristics of the students and families being served. Educators, 
students, and families can use multiple resources (e.g., observation checklists, surveys, and self-assessments) 
to establish the format and select the criteria for the assessment tools. Figure 4 presents some sample 
curriculum and instruction assessment instruments. 

Collecting assessment data to plan instruction and build curricula can be facilitated through the use of 
additional evaluation tools such as descriptive guides and forms for judging student learning, anecdotal 
records, tcachcr-parcnt-studcnt conference interviews, learning logs, observation forms for viewing audio/ 
videotapes, and photographs. v Such multiple indicators allow educators to develop a clear picture of the 
effectiveness of any curriculum or instructional practice. 

Staff 

The school staff play a critical role in implementing a whole-school bilingual education program. 
Development of an effective curriculum coupled with appropriate instructional practices requires a great 
deal of coordinated staff effort. The review of effective school staff practices identified the following four 
general characteristics of staff qualities: broad knowledge and experience base, an awareness of inservicc 
training needs, a sense of autonomy, and the ability to collaborate. Careful thought must be given to the 
assessment of these and other related staff qualities. Ultimately, the assessment of staff development must be 
linked to student learning. 

Knowledge and experience . Lucas (1993) indicates that staff need to be knowledgeable about issues 
related to language minority education. But what constitutes "knowledgeable" and what are the issues? In 
order to address these questions, the school staff first must reflect on and identify their key issues; they then 
can make a valid determination concerning the match between their expertise vis a vis the needs of the 
students. Reflection on student needs must be ongoing since student populations arc changing and their 
needs rarely remain static. Consequently, the staff knowledge base and experience must continually evolve. 

Another area of concern is the varied depth of staff experience and knowledge (e.g., awareness of the 
issues, ability to apply knowledge, and ability to train other staff). Although it is unrealistic to expect all 
staff members to be trainers on all issues, all staff should at least have a threshold level of knowledge (i.e., an 
awareness) of key language minority issues and move toward developing expertise in one or more related 
areas of educational priority (e.g., language development, instruction, or assessment). 

The key features for examining the effectiveness of stuff knowledge and experience include: the degree 
of awareness the staff have of students' needs; the degree to which the staff have moved towards acquiring 
the desired level of expertise in targeted knowledge areas; and the degree to which staff knowledge and 
experience continue to evolve. 
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Training . A critical link to continuous growth of staff knowledge is staff training. An active, productive 
staff training component characterizes effective schools serving LM students (e.g., Garcia 1993; Lucas 
1993; Lucas, Hcnzc, and Donato 1990; McCollum, and Russo 1992; Tikunoff 1991). Staff inscrvice 
training must be based on meeting the needs of all students. Clearly, if the staff have examined the needs of 
their students and their own knowledge bases carefully, the selection and prioritization of inservice training 
should be evident. Moreover, inscrvice training should be ongoing and coupled with practice and coaching. 

Autonomy . The third staff feature associated with effective schooling concerns the issue of staff 
autonomy or feelings of empowerment to make decisions (c.f. Gonzalez ct al. 1993). At a minimum, 
informed decision-making might include making explicit the nature of the problem and possible solutions, 
consultations with other teachers and experts, and developing a plan of action and a mechanism to assess 
the impact of the action taken in the first place. 

Evidence of staff autonomy can occur at different levels — from individual student, small group, and 
classroom levels to grade and whole-school levels. Traditionally, teachers have had the most autonomy at 
the individual student, student group, and classroom levels. If the decisions made at these smaller levels are 
well informed, staff autonomy at higher levels of organization (i.e., the grade and whole-school levels) 
should be encouraged. Teams of experts could make decisions at most school levels. 

Garcia (1993) maintains that, within effective schools, staff also should have a sense of self-efficacy and 
see themselves as innovators. This offers staff opportunities to expand their knowledge and the autonomy 
to apply it. 

Collaboration . The final effective whole-school feature concerning staff development is collaboration 
(McPartland and Braddock 1993; Garcia 1993). This feature can be defined as coordinated sharing of 
resources such as knowledge, support, staff, materials, and decision-making autonomy. Effectively identi- 
fying student needs, working within and across teams of staff experts, and negotiating needed inservice 
training, all require active collaboration among staff. 

Assessing staff development . The use of self-assessments, peer reviews, focus groups, objective outsiders, 
and the students themselves will generate different and valued perspectives on staff development. A variety 
of assessment strategics (e.g., interviews, surveys, questionnaires) will aid in generating a range of evidence 
essential for making a valid judgment on the quality of staff development. Figure 5 provides some sample 
assessment formats and questions related to staff development. 

In summary, if a staff is to implement a whole-school bilingual education program that will serve both 
monolingual and bilingual students equitably, it will have to engage in a dynamic process of well 
coordinated development activities. 

Administrators 

A promising aspect of the nations efforts to restructure its schools is that an increasing number of 
people who are interested or involved in the education of LM students appears to be reaching consensus on 
an array of issues. One point in particular suggests that successful restructuring will depend largely on the 
ability of people at all levels of the education system to change (Olscn et al. 1994). 

One facet of bilingual program implementation where this especially holds true is at the administrator 
level. A review of the effective schools research conducted over the past several years acknowledges that 
administrators play a critical role in facilitating program implementation that positively affects language 
minority students' outcomes (de George 1991). Identifying specific roles as well as assessing their impact 
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Figure 4 

Sample Curriculum and Instruction Assessments 



Sample Curriculum Observation Checklist 

Check (/ ) any of the following features observed in your schools curriculum* 
3 Native language development 

□ ESL classes use approaches appropriate for the ESL level of the student: 

Total Physical Response 

Natural Language Approach 

Language Experience 

J Content classes in native language and English 

Sample Teacher Native Language Use Rating Scale 

Rate the use of native language on the following scale: I = unsatisfactory; 2 = satisfactory; 3 = outstanding. 

} , Students' native language was used in a meaningful context. 1 2 3 

2. The teacher \s use of students' native language was understandable. 1 2 3 

3. The teacher was able to understand students* questions, responses, 1 2 3 
and problems. 

4. The teacher checked students' comprehension during instruction. 1 2 



Sample Text Review 

Circle the response that most closely matches what you see in the text material 

1. Information is accurate. 

2. Graphics and illustrations reflect students' culture and background. 

3. Reading level is developmental^ appropriate for the target students. 

4. The overall rating of this text is: {Circle one) 



Yes 
Yes 
Yes 



No 
No 
No 



Excellent Good Adequate Poor 



Sample Student Survey Questionnaire 

(Can he developed in native language and English as appropriate) 

Rate each item according to the following scale: SA = strongly agree; A = agree; D = disagree; 
SD = strongly disagree. 

1 . My classes are interesting to me. 

2. I am involved in the planning of classroom activities. 

3. There is a variety of reading materials available in my first language. 

4. The reading materials available are useful and interesting to me. 



SA 


A 


D 


SD 


SA 


A 


D 


SO 


SA 


A 


D 


SD 


SA 


A 


D 


SD 



Sample Teacher Peer Observation or Teacher Self-Assessment 

Rate each statement aecording to the following scale: 3 = always; 2 = sometimes; I = m 

1 . Does the lesson move from the general to the specific? 

2. Are details presented within a general conceptual framework? 

3. Is there an attempt to draw on student background knowledge and interests? 

4. Are students given choices? 
Is the content meaningful? 

6. Is there support for the students' first language and culture (even if the 
teacher does not speak the students' native language)? 



2 
2 
i 



3 
3 
3 
3 
3 
3 
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on overall student and program accomplishments are critical factors in determining whether whole-school 
bilingual education programs are implemented effectively. As Segan and Scgan (1991) propose, the 
institutionalization and acceptance of bilingual education as an integral part of the school depends on both 
the support that administrators provide and their ability to communicate to all staff. Features of effective 
bilingual programs that focus more specifically on the roles of administrators and their practices have been 
identified by Garcra ( 1 987), de George ( 1 99 1 ), and Segan, and Segan (1991). These effective features can 
be categorized into three areas: understanding and knowledge, collaboration, and support. 

Understanding and knowledge . It is imperative for all administrators to develop an extensive knowl- 
edge base regarding bilingual education. Dc George (1991) suggests that the knowledge base could be 
clustered into three areas: student and staff needs, theory and practice of bilingual education, and culture 
and cross-cultural communication. Key features of the administrator's knowledge base include: 

developing an understanding of the history of bilingual education including legal, social, and political 
implications; instructional theories; implementation models; and current issues and trends; 
recognizing the challenge of teaching LEP students, the need for ongoing and consistent support for 
staff, and the need for many resources; 
using multiple assessment techniques; and 

confirming the importance and value of cultural diversity and promoting language and culture as 
resources to be maintained. 

Collaboration . Implementing effective bilingual programs involves entire school communities and 
shared decision making. Implementing bilingual programs that permeate every aspect of the school or 
district will mean new dynamics and structures of administration and management that focus on team- 
work, collaboration, building trust, and working together (Olsen et al. 1994). According to de George 
(1991), Rosenholtz (in Scgan, and Segan 1991). and others, key features in administrators' collaborative 
efforts include: 

communicating high achievement goals for all students and organizing action that links family, 
student, and teacher efforts; 

engineering opportunities for teacher collaboration at many levels as part of a comprehensive team- 
building plan; 

encouraging joint administrator-teacher decision-making in instructional matters and enhancing the 
articulation of each classroom with an effective bilingual program; and 

managing relationships with local, state, and federal external groups, agencies, and organizations. 

Support. In districts where effective bilingual programs are implemented, there typically is a high 
degree of administrative support from the superintendent's level down. Effective bilingual programs benefit 
from administrators who demonstrate high levels of advocacy for the program, its students, and its staff. 
1 he key features of an administrator's supportive efforts include: 

recruiting staff at all levels who are not only sensitive hut knowledgeable about issues related to 
bilingual education and language minority students; 

creating a vision for the bilingual education program that includes successful outcomes for students, 
staff, and parents; 
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promoting a salary differential policy for teachers who are endorsed or licensed in bilingual/ESI. 
education in recognition of the importance of their specialization; 

promoting and providing comprehensive staff development which includes pre- and inservice training, 
credit unit equivalents, and university/college coursework; and 
ensuring that the needs of I. HP students arc addressed. 

Assessing administrative efforts . The successful implementation of a whole-school bilingual education 
program will depend heavily on the quality of leadership provided at all administrative levels. A paradigm 
shift in the national system of education is creating a need for leadership vitality in bilingual education that 
stresses flexibility, collaboration, and the development of a comprehensive knowledge base. Figure 5 
provides sample assessments that focus on the roles and functions of administrators. 

FAMILY INVOLVEMENT 

Traditionally, successful family involvement is measured by theincreased active participation of parents 
in schools or programs. The challenge faced in linguistically diverse communities is that the typical 
expectation of school involvement docs not accommodate the needs of those families facing the barriers of 
poverty, illiteracy, and cultural and linguistic differences (Goodson, Swartz, and Millsap 1991). 

Based on numerous studies of exemplary bilingual and multilingual family programs (e.g., Cummins 
1986; Faltis and Merino 1992; Lucas 1993; McCollum and Russo 1992), a new framework is proposed for 
designing and assessing the effectiveness of family involvement in multilingual communities. This frame- 
work reflects three common features: outreach, collaboration, and support. 

Outreach efforts expand involvement from the parent to all family members; attempts are made to 
address the long-term needs of children by serving the short-term or immediate needs of the whole 
family. 

Collaborations are established among key stakeholders (community leaders, educators, parents, and 
students) from the onset. All are involved in decisions ranging from scheduling to setting policy. 
Support is provided through activities and services that will impact student learning directly and 
indirectly. By communicating to families in their home language, providing English-language instruc- 
tion, and offering a wide variety of activities and services, schools can promote active parental 
participation. Parent advocates, counselors, or liaisons who arc familiar with the community culture 
and who can speak the languagc(s) of the community can be employed to train school staff and bridge 
community school relations. 

Assessing family in volvem en t . In determining an assessment system for this innovative concept of 
family involvement that is practical and useful, we recommend identifying the needs of parents, assessing 
the impact of the collaborations and partnerships, and monitoring and assessing the effectiveness of family 
involvement activities. Specific examples of assessment items and formats for each step are provided for 
each step in Figure 6. 

Other assessment devices that can be used to monitor numbers of persons participating in activities as 
well as the impact activities are having on families include sign-in sheets, self-reflections, diaries, logs, and 
focus groups. Family literacy skills can be assessed with various types of instruments including standardized 
literacy tests, interviews, observation devices, self-assessments, and performance samples (Holt 1994). 
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Figure 5 

Sample Assessments for Staff and Administrators 



Sample Interview Questions To Establish Staff Knowledge 

1. What are the most pressing needs of your students? What evidence supports your opinion? 

2. Which of these needs do you feel the staff have the knowledge and expertise to effectively 
address? 

3. In which of these needs areas would you like to develop greater expertise? 

4. Would you be interested in providing peer-training in any one of the needs areas you've 
identified? 

Sample Rating Scale After Inservice Training 

Rate the following statements on a scale from (I) - lowest level of agreement to (4) - highest level of 
agreement with the statement. 

1. Previous staff discussions prepared me for this training. 12 3 4 

2. I feel I have a better understanding of the issues. 12 3 4 

3. I integrate this skill into my repertoire of teaching strategies. \ 2 3 4 

4. I feel the administration has facilitated my participation in this training. 12 3 4 

Sample Questionnaire Items After Practicing Skill 

Please respond to the following questions. 

1 . How effective was the peer-coaching you received? 

2. If you have begun to systematically apply the new skill, have you noticed any impact on students? 

3. What kind of additional training or activities in this area would be useful to you? 

4. Has the administration been supportive of your efforts to develop new skills? 

Staff survey with Likert-scale responses 

Circle the response that describes your building administrators: most of the time (Ml), sometimes (S), or 
rarely (R). 

1. Joint administrator-teacher decision making is encouraged. MT S R 

2. Opportunities for comprehensive staff development are consistently provided. MT S R 

3. The bilingual program is viewed as an integral part of all school programs. MT S R 

4. Parents are welcomed to school and treated with respect. MT S R 



Selected Self-Assessment Survey Items for Assessing Administrators 

1. Do you encourage shared decision making? Yes No 

2. Do you consistently communicate high achievement for all students as a goal? Yes No 

3. Do you promote bilingual ism and bi lite racy as something valuable for all students? Yes No 

Open-Ended Interview With Administrators 

1. What are the goals and objectives of your school's bilingual program? 

2. How could teacher-administrator collaboration in the school be enhanced? 

3. Are you familiar with the legal implications and requirements that apply to l.KP students? 

4. In what specific ways do you model the importance and value of cultural diversity? 

Focus Group Questions for Administrators and Staff 

1 . What is the district's vision of bilingual education? 

2. Are goals and objectives clearly articulated? 

3. What can administrators do to promote bilingual education in the community? 
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Figure 6 

Sample Formats and Items for Assessing Family Involvement 



Selected Survey Items for Assessing Family Needs 

Check the areas you would like to receive information about or training in from the school. 
Children Myself 

□ Helping children with homework □ Improving English speaking/ reading/writing skills 

How to discipline children ul Studying for a GHD 

J Provide counseling for children CJ Improving talking with my children 

Selected Interview Questions for Assessing Family Needs 

1 . What kinds of help do your children need? In school? At home? 

2. What would you like to learn to help your child? At school? At home? 

3. What time is the best for you to attend classes or meetings that may be developed? 

Selected Survey Items for Assessing Family Participation and Satisfaction in Partnership Activities 
Circle the number that best describes your participation and satisfaction with our partnership activities. 
Key; 1 - Low; 2 = Mediocre; 3 = High. 

Participation Satisfaction 

1. Planning goals 12 3 12 3 

2. Designing program activities 12 3 12 3 

3. Assessing family needs 12 3 12 3 

Selected Approaches for Assessing Family Activities 

1. Sign-in sheet information : Include the following information: date of activity; name of activity; 
name of participants; names of children, if attending; and sponsor(s) of the activity (e.g., Title I, 
VII. Indian Education, iMigrant Education). 

2. 1 n te rv i e w/ O ucs t i o n n a i re Res po n ses : Questions to ask: In which activities are you participating? 
What part of the activity sessions do you like best? What have you learned in these sessions? What else 
would you like to learn in the activity sessions? 

3. S e 1 f- assess m e n t s/ re fl ec t i o n of skill dev e 1 o p m e n t : Using video- or audiotapes, self-rating forms, 
self-reports, ind comparisons of their own techniques/skills with modeled performances, parents 
can ask themselves the following questions: What skills or techniques am I learning? hi what areas 
am I strong? How do I know I am strong in those areas? In what areas do I need improvement? What 
do I need to do to improve in those areas? 

4. hocus groups questions : What family literacy activities are most use fit I? Why? What activities could 
we enhance or delete? What other activities do you feel would be most usefid to the community? Who 
would benefit fiom these activities? 

5 . f amily literacy assessments : Prc/post-assessmeuts usi ng a co m b i n at io n of m ul t i p le-ch o ice sho rt essay 
questions to measure reading comprehension and writing skills. Topics can range from cooking 
and nutrition to the U.S. school system, as long as they are tied to the adult literacy framework of 
the program. 

Observations of adult listening, reading, speaking, and writing skills can be rated according to the 
following scale: A = with ease; 3 = with some support; 2 = with great difficulty; 1 = not at all. Skills 
can also be assessed holistically or analytically based on predetermined criteria (e.g., writing criteria 
can be developed for the following rubrics: voice, organization, word choice, and mechanics). 
Other rubrics can consist of beginning literacy; developing literacy, and full literacy. 
Portfolios: Can be used to gather a complete picture of individual family members' participation 
and performance. 
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In short, an assessment plan for family involvement programs should be viewed as a dynamic process. 
It is linked closely to the diverse needs of families within the community, to the collaborative efforts of all 
key stakeholders, and to the activities and services required for successful family participation. 

Program Implementation Summary 

The essential component for a whole-school bilingual education program is the effective implementa- 
tion of an appropriate and sensitive curriculum. The curriculum must be sensitive to cultural, ethnic* and 
linguistic differences and must support learning in the native language as well as in English. For this to be 
effective, staff must have the appropriate knowledge and experience, and must be empowered to ucilizc 
them. Administrators must understand the purpose of and support all academic programs within the 
school and must ensure collaboration both within and outside of the school. Finally, the entire family 
should be involved in the student's education. Once program context is in place and program implementa- 
tion has begun, student outcomes can be considered. Without notable student outcomes, the program 
cannot be said to be truly successful. 

STUDENT OUTCOMES 

If the schools program context and program implementation are in place, then equitable and positive 
student learning are possible. Traditionally, student outcomes focus on achievement in the standard 
content areas of math, science, social studies, and language arts. In effective bilingual education programs, 
other essential results have been identified as well. The acquisition of learning strategies and improved self- 
concepts are as necessary to student growth and progress as academic achievement. This section describes 
learning strategics that can be used to promote positive academic achievement; self-concept strategies, 
including motivation, confidence, and ethnic self-pride; and assessment approaches that can be used to 
measure student progress and accomplishments. 

Learning Strategics 

Learning strategics are conscious mental processes that students develop and use in new learning 
situations, and to assess goals and outcomes. Students will learn more when they arc able to reflect on what 
they are learning, and internalize and understand how they learn best. In other words, positive student 
outcomes (achievement) can result from the development of reflective cognitive monitoring and the 
development of metacognitive skills. Research on effective whole-school programs for LM learners 
identifies the particular advantage that bilingual students have in the area of metalinguistic awareness. For 
example, bilingual students often possess the ability to analyze the form as well as the content of language, 
the knowledge of how to talk about language, and control over non-literal uses of language such as puns, 
irony, and figures of speech (Snow 1990). Teachers are in an ideal position to model strategies that can 
enhance these abilities; they can demonstrate to students the thought processes and behaviors involved in 
applying metalinguistic and metacognitive skills. 

The goal for any whole-school program is to prepare independent and cognitively aware students. In 
order to do this, students must participate actively in practice tasks and in processing new information into 
prior information, and must apply higher-order thinking skills in the target languages. Each of these areas 
is discussed in the following section along with methods for assessing whether these are occurring within 
the whole-school bilingual education program classroom. 
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Practice . Students require opportunities to practice applying learning strategies through active involve- 
ment in meaningful learning. Students who develop an awareness of learning strategics that can be applied 
to various tasks are better equipped to exhibit high levels of productive and successful task engagement. It 
is especially important for LM students to practice the application of learning strategies in activities that are 
relevant to them and that stem from their own strengths and diverse backgrounds. Language minority 
students also require ample opportunities to practice and apply learning strategics in their native language 
so that they then can transfer or apply their knowledge or these specific learning strategics to English task 
demands. 

Information processing . All learners benefit greatly when new information is built upon prior knowl- 
edge and abilities. Positive outcomes can result from students' participation in schema building activities 
and the application of knowledge across content areas. Schema building activities such as webbing, quick- 
writes, and discussions, allow teachers to identify what students know about a topic. Subsequent opportu- 
nities to process information can build or. existing strengths and include hands-on, process-oriented, 
meaningful activities. It is important that students have these opportunities to use new language skills to 
draw connections between the second language and concepts in their native language. 

Higher order thinking . To develop students' attitudes, achievement levels, and abilities in two lan- 
guages, offer meaningful active involvement across content areas in higher order thinking skills such as 
inquiry and problem solving. The ability to transfer skills to new learning situations and real life situations 
is enhanced through die development of higher order thinking skills. Conscious use of effective learning 
strategics in the native language allows students to transfer those skills to a second language. Students and 
teachers both benefit from opportunities to reflect on and evaluate the quality of their own thinking skills 
and strategies (Gardner 1993). 

Assessment of learning strategies . Effective assessment of learning strategies involves determining what 
strategies are being used then looking for positive achievement and self-image outcomes. The easiest way to 
ascertain the learning strategies being used is through observational checklists; in addition , students can be 
asked questions about their perceptions of what they are learning. Assessment in this area is not graded, but 
is utilized to determine how rather than what students learn. Samples of assessment tools directly related to 
learning strategies and self-enhancement are included in Figure 7. 

Self Enhancement 

The objectives of many education systems sometimes involve little more than learning specific content 
and procedural skills. However, research on excellent language minority programs identifies other 
outcomes such as the development of positive cultural and ethnic self-concepts, motivation to learn, and 
self-confidence as objectives of the education process. Self-concept is defined as skills and attitudes, 
including aspects of temperament and personality, that play a significant role in the individuals identity 
and engagement in the learning process. Self-esteem reflects a value judgement about oneself and ones 
morals, principles, and spirituality. Self-enhancement occurs when language and culture are validated 
through inclusion in all aspects of the whole-school bilingual education program and when students see 
themselves as an integral and constructive part of the learning environment. 

CnltnralJethn ic iden tity . Many language minority programs include development of a strong, positive 
ethnic identity as an objective necessary for the advancement of traditional academic achievement objec- 
tives. Academic achievement often is less-than-optimum for students who identify themselves as "differ- 
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Figure 6 

Sample Assessments for Learning Strategies for Self Enhancement 

Procedures for Conducting a Focus Group to Find New Strategies to Motivate Students 

Participants (between 5 to 10) include seventh grade teachers and education assistants. 
■J The purpose of the group is to identify strategies that actively involve students in the learning 

process and motivate them to enjoy learning. 
_J The group meets monthly to brainstorm new ideas and report on the effectiveness of strategies. 
— I A note book or ideas and comments on the effectiveness of ideas is kept and made available in 

the staff lounge. 



Sample Student Learning Strategies Self Evaluation 

Today in Social Studies... 
) . I learned more about... 

2. I still have the following questions: 

3. I have developed the following plans to answer these questions... 



A Sample Rating of a Student's Writing Strategies 








Does the student. . . 








1 . use constructive strategies for getting drafts started? 


always 


sometimes 


rarely 


2. take the conference period seriously? 


always 


sometimes 


rarely 


3. help peers by listening to their drafts? 


always 


sometimes 


rarely 


4. use revision and editing processes? 


always 


.sometimes 


rarely 


use support systems in the classroom (manuals* 








guidelines, spelling aides, resource materials)? 


always 


sometimes 


rarely 


6. actively participate during sharing? 


always 


sometimes 


rarely 



Sample Ethnic Pride Self-Assessment 

Students complete statements about ethnic pride at the beginning of the year and again at the end of the 
year. 

1 . Understanding my own traditional customs helps me appreciate others by... 

2. Speaking my family's language is important because... 

3. Here's what I do to show pride in who I am... 



Sample Motivation and Confidence Self-Assessment 

Circle the response that you feel most closely matches your feelings about your work in this class. The 
responses are (I) rarely. (2) sometimes, and (J) often. 

1. I work hard in this class. 1 2 3 

2. 1 understand the material for the class. 1 2 3 

3. This class is difficult, but worth the hard work. 1 2 3 

4. 1 speak up in class and have good ideas to share. 1 2 3 

5. Other students in class look up to me. 1 2 3 
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em" and who feel this difference i-s equated with inadequacy. In our school systems, children who feel 
"different" arc often children who do not speak English as their native language, or others who can be 
identified easily based on appearance (handicap, racial/ethnic, and so on). Cultural/ethnic identity is 
defined as a positive sense of self that radiates from an awareness of the beliefs, values> customs, and 
language of the culture to which an individual is born and raised. 

Motivation . Motivation can be defined as the incentive to learn when the context for that learning is 
inclusive. Motivation to learn has long been acknowledged as a critical factor in an individuals ability and 
willingness to engage in the learning process. The effective schools research indicates that children are 
motivated to learn when the materials, methods> and content have personal meaning for them. For LM 
children, meaningfulness can be translated into the inclusion of their values, beliefs, customs, and language 
in all aspects of the instructional delivery system. 

Confidence . Confidence is defined as assurance and conviction or self-possession; it is enhanced for LM 
students in an inclusive learning environment. Confidence is strongly tied to self-evaluation, which is 
dependent on two primary sources of information: the individual s experience, and what the individual 
thinks other people think of him/her. When LM students find themselves in education systems that reflect 
their learning styles and strategies and that validate their culture and way of life, confidence in ability to 
succeed is often a positive outcome that facilitates academic achievement. 

Assessment of self-enhancement . The assessment of learning strategies and self-enhancement rarely 
occurs through the use of standardized or norm-referenced tests. Instead, surveys of attitudes, observations 
of behavior as recorded through checklists and interviews, and self-assessment of attitudes and skills can be 
useful tools for measuring change/progress. Assessment is usually not graded, so the importance and 
relevance of self-enhancement to academic learning must be underscored in some other fashion. Figure 6 
provides some examples of assessment instruments used to measure self enhancement. 

Student Performance Assessment 

Assessment of student performance is important for at least two reasons: it ensures that students are 
improving their content and linguistic knowledge; and it "proves" to funding agencies that the special 
programs they funded are working. In the past, standardized tests have been the primary means for 
measuring what students know, not only for mainstream projects but also for federal:/ funded programs 
such as Title I. Since the Nation at Risk report (National Commission on Excellence in Education 1983), 
there has been talk of reforming assessments and assessment practices. Key to the current reform movement 
in the Clinton administrations Goals 2000 legislation are the three types of standards identified: content 
standards define what should be taught and what students should learn; perfonnance standards are 
concrete examples and definitions of what students should know and be able to do; and opportunity to 
learn standards define the criteria for assessing the adequacy of the learning opportunities provided for 
students. 

The objective now must be to determine appropriate performance standards for students within whole- 
school bilingual education programs. To do so, both standardized tests and alternative assessments must be 
considered, including the purposes for which they function best and for which they arc appropriate within 
a whole-school context. 

Alternative assessments . The current reform movement in education and education testing is forcing 
many psychometricians to rethink their position on assessing students. Stakeholders arc beginning to 
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suggest new frameworks that move away from testing students with the traditional standardized tests (e.g., 
Berlak 1992; Figucroa 1990; Gardner 1993). Although the manner in which we should assess students is 
not yet described fully, there is some basic agreement that the goal is con textual ized measurement of 
student achievement. Gardner (1993) describes such assessment as ecologically valid, intelligence-fair, 
sensitive, intrinsically interesting and motivating, and part of the natural learning environment. Not only 
do alternative assessments measure students' knowledge, but they also can measure specific skills. 

Alternative measures can take many forms, depending on the content and purpose of the assessment. 
The use of alternative assessments can be linked easily to classroom routines and learning activities. Some of 
the forms of alternative assessment that are more specifically appropriate for assessing students' learning 
include anecdotal records, teacher and student checklists, peer reviews, questionnaires, holistic rating 
scales, quizzes, criterion-referenced tests, cloze tests, and mi "cue analyses. Some forms more closely linked 
to activities include: 



J essays, reports 

□ journal entries, logs 
J collaborative work 

□ homework 



J poetry, creative writing 

□ posters, artistic media 
J reading lists 

□ games 



-J story retelling 

□ brainstorming 
sJ writing samples 

□ debates, presentations 



(from De Fina 1992, and Navarrete et al. 1990) 



Although many of these alternative assessments appear to focus on student achievement, it should also 
be pointed out that they can be used to assess language proficiency (both native language and English). 
With alternative assessment, a single instructional activity can generate measures of language proficiency 
across all four modalities (reading, writing, speaking, and listening). Essays, reports, letters, and journal 
entries can be used as measures of the students ability across different genres of writing. Similarly, reading 
lists, book reports, and story retelling can generate measures of die student's comprehension and reading 
ability. Observations of games, debates, and conferences can give indications of the student's oral and aural 
proficiencies. 

Because alternative assessment can and should take many forms, teachers are frequently left with a 
major question: What do 1 do with all of this? How can it be organized to make sense of it all? One 
approach to organizing the information is through a portfolio. Portfolios can serve as a repository for a 
student's "best*' work and to demonstrate achievement and readiness to progress through the curriculum or 
program. By looking at "best" products across the school year, growth is demonstrated. Another way to 
demonstrate growth is to use the portfolio as a process or a place to keep all student work on a particular 
project, from first notes to final draft. In this way, growth can also be seen as the child, teacher, parents, 
and, possibly, peers evaluate the project s progress and the child modifies and edits the work. Portfolios can 
be empowering, can provide important information for school staff, and can show achievements in a 
practical manner. However, certain key elements must be in place when attempting to utilize portfolios. 

1. Portfolios must be systematic, purposeful, and meaningful; they must have a clear purpose. The 
portfolio is not just a collection of work; each piece must be selected carefully to demonstrate growth 
and achievement — it must have a reason for being in the portfolio. Also, the portfolio can be used to 
collect materials within or across content areas. 
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2. Portfolios must exhibit self-reflection and judgement. In choosing the works to be included, teachers 
and learners should set guidelines and criteria. 

3. Portfolios must interact with the curriculum. Portfolios must contain work that demonstrates a variety 
of tasks measuring the same skill and must demonstrate a close link between those tasks and curriculum 
objectives. 

4. Portfolios must be assessed reliably. It is important that criteria be set so that administrators, teachers, 
parents, and/or students can all agree on what is a good portfolio. 

This last clement — portfolio assessment — is a controversial issue. Some argue that portfolios should 
not be graded, although individual contents should be graded. Others argue the opposite. First, it is 
important to recognize that the grading issue is based, in part, on the purpose of the portfolio. If the 
purpose is to meet the mandates of a funding agency to show "success," then grading of at least parts of the 
portfolio will be necessary. 

When looking at individual pieces of work, it should be remembered that not every one need be 
graded. For some assignments, it may be enough to note that the work was completed or that the child 
participated, rather than having to assign a specific grade or score. At other times, a specific scoie may be 
needed. In such cases, scoring procedures such as holistic, primary trait, and/or analytic, should be 
considered. 

For any type of scoring procedure, a method, or "rubric," should be developed to delineate the criterion 
for each score. For instance, if a holistic score of one to four is utilized, what constitutes a "one" and what 
does a student need to do to demonstrate "four" skills? Both time and practice are necessary to develop a 
sound, reliable scoring method. Other scoring methods arc also appropriate for different types of alterna- 
tive assessments. It is important to remember, however, that the criteria developed must reflect sound and 
reliable practices. 

Standardized tests . When referring to standardized tests, most people think of norm-referenced tests 
(NRTs). NRTs are typically used to sort people into groups based on their assumed skills in a particular 
area. Although NRTs are designed to allow for comparison of students across grades, cities, states, and 
countries, such comparisons may not be appropriate for whole-school programs that arc truly sensitive to 
all their students' reeds. 

NRTs can be ireful to the teacher when either (1) the objectives of the curriculum being used match 
the objectives of the NRT, (2) the classroom teacher can access item scores for each student, or (3) the same 
NRT is used for needs and outcome assessments. The former, the match of curriculum and NRT, is 
important because the extent of the match will give more or less credence to the students' scores on the test. 
For instance, if the match between NRT and curriculum, referred to as "coverage," is good and students do 
poorly on the test, then teaching strategies, program objectives, and other factors should be reviewed; if 
coverage is poor, it should be expected that the students will score poorly. If item scores are available, the 
teacher can review error patterns to determine which items and portions of the test were problematic for 
students (e.g., the visual recognition portion of the reading subtest). However, this is time consuming and 
only possible if the school requests the necessary scoring information from the test scoring company. 
Finally, using the same NRT on an annual basis will allow comparison of students to baseline information 
and to national, regional, or local norm groups. 
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There also arc some cautions to be aware of when using NRTs. Those most applicable to a whole- 
school bilingual education program include: 

1 . NRTs were designed to determine how well a student does in comparison to a particular norm group; 
they are not designed to show how well the student knows a particular subject area; and 

2. they have been used to assess progress in specially-funded programs on the basis that these students 
should become "more like" their mainstream peers. For example, a LM student should learn to write 
with the same skill as an English-only peer. If this is an appropriate comparison, at what point in time 
docs this expectation become realistic? 

Commercially available language proficiency tests are considered standardized tests since they arc 
administered in a uniform manner and generate results that compare the individuals test score to a norm 
group s. These types of language proficiency tests can be useful for making a number of language-related 
decisions (e.g., placement, monitoring language development). Similarly, test data generated from these 
types of tests are often useful for reviewing the impact a whole-school bilingual education program has had 
on language proficiency development over time. 

However, as with any psychometric instrument, there are a number of precautions one should take 
when using standardized test scores to make a judgement regarding a students language proficiency. 
Currently, many of the commercially available language proficiency tests arc discrete-point measures of 
grammatical competence. Language proficiency entails much more (e.g., discourse organization, language 
functions, non-verbal communication) than just grammatical competence (Bachman 1990; Oiler 1991). 

In addition, while recent attempts have been made on the part of standardized language test developers 
to capture the linguistic processes as they present themselves in a school setting, they arc not fully successful 
cither. The four skills (aural, oral, reading, and writing) on these commercial language proficiency tests 
gcnerallv are not integrated in a lifelike manner. Thus, while language proficiency tests can be useful, they 
must be supplemented with additional evidence that is critical before a valid judgement or decision can be 
made regarding a student's language proficiency. 

Student Outcome Summary 

Frequently the only measure of program success is student achievement. Though student performance 
is important, the methods for measuring it must be cautiously considered. Careful attention must be paid 
to how assessments measure both achievement and language proficiency and how the skills are actually 
called upon in a real-life school setting across the curriculum. Standardized measures, especially commer- 
cially available tests, must be supplemented with additional evidence before valid and reliable judgements 
about a students content area achievement or language skills car. be made. 

In addition to academic achievement outcomes, the effects on self-image must also be considered. The 
whole-school bilingual education program should seek to ensure that students' ethnic pride, confidence, 
and motivation levels arc enhanced and that various learning strategics that are appropriate for ethnically 
and linguistically diverse students are employed. This section has described student assessment, in general, 
and, more specifically, the assessment of learning strategies and self-enhancement. 
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A COMPREHENSIVE EVALUATION PLAN 

A fully developed assessment system is an essential component of any whole-school program. However, 
a recent study on school restructuring in California found that "schools are failing to consider information 
that will help them plan appropriate programs or assess the effectiveness of their reforms for students of 
different linguistic, cultural, or racial groups" (Olsen et al. 1994, p. 20). A concomitant problem is the lack 
of resources and support available to schools for the design and utilization of new assessment procedures 
(Olsen et al. 1994). In this section we suggest components of an assessment program that will assure that 
the community's needs are addressed by the school, that key components of program context and 
implementation can be measured, and that the effectiveness of the curriculum (i.e., learning) can be 
evaluated; in addition, methods for assuring the meaningfulness of the assessment practices are introduced. 

Creating an Effective Assessment and Evaluation System 

The overall purpose of a whole-school programs assessment system is to initiate and maintain 
discussion about how the school addresses the educational needs of all students, including LM students. In 
so doing, the school must be prepared to assess each of the components of school context and program 
implementation, as well as student outcomes. T he objectives of each clement of the assessment system 
must be understood clearly by school staff and by students and their families. In general, the elements of an 
assessment system include: 

a needs assessment to determine the current status of the indicators of school context, program 
implementation, and student outcomes. "Initial assessment is important in the development of a 
successful [whole-school] program because it provides program planners with the opportunity to 
examine the needs, desire, and goals of the potential participants" (i.e., the entire school) (Graham, in 
Holt 1994, p. 36). The basic questions are, "Where arc we now? What is the school doing?"; 
measures of progress to determine the successful features of the program, the shortcomings of the 
program, and whether program implementation and the students arc progressing in the expected 
manner. Assessing the progress "can help staff determine the extent to which their instructional efforts 
lead to beneficial results and. ..give participants a sense of accomplishment by showing them that they 
arc making headway" (Wriglcy, in Holt 1994, p. 61). The basic question is "How much change has 
there been from the beginning of the program until now?"; 

outcome measures to determine whether the objectives of the academic year or educational unit have 
been met. These measures make it possible to summarize the progress made by the whole-school 
bilingual education program during the entire academic year. The basic question is, "How much 
change did we effect this year?"; and 

evaluation an aggregation of progress and outcomes measures to determine whether the needs 
originally identified have been met by the program. In other words, "What changes have been made in 
school context, program implementation, and student outcomes?" 

Designing the Evaluation 

Just having data collected from various measures of progress and outcome is not enough. Data must be 
organized in a meaningful way to ensure that the conclusions about the success of the program are explicit 
and useful. To do this, evaluation design must be carefully considered prior to program implementation. 
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Two general types of information have been described as useful within a school program. The first, 
formative evaluation, is conducted at various points during the planning and early stages of the program. A 
good formative evaluation can identify problems and suggest ways to modify the program in order to 
improve it. Thus, the needs assessment should be repeated during the early stages of the actual program to 
ensure that the population being served has not changed and that the identified needs still are germane. 
The use of dynamic evaluation has increased in recent years. While similar to formative evaluation, 
dynamic evaluation is more ongoing in nature and can provide suggestions for improving the project 
throughout its life (c.f, Figucroa 1990). Although some education agencies may require written formative 
or dynamic evaluation reports, frequently oral reports to key stakeholders in decision-making roles may be 
more appropriate. 

The second type of evaluation, summative evaluation, tends to be more formal. This might be the 
year-end report or the end-of-project report. The summative evaluation generally takes all the assessment 
results from across the life of the program (or the year), summarizes them, provides information about the 
strengths and weaknesses of the program, and suggests methods for further improvement in upcoming 
years. The summative report generally is written for, and frequently must follow, the particular require- 
ments of a funding agency. 

It is often best to consult with a professional evaluator when planning an evaluation design. There can 
be many pitfalls in designing and utilizing assessment instruments, especially for a bilingual population. In 
addition, many funding agencies have specific requirements and regulations that must be followed. A 
professional evaluator may have ideas on how these requirements can be followed while also serving the 
needs of the community. 

Ensuring Meaningful Measurement 

To ensure that an assessment is meaningful for the whole-schooi bilingual education program, two 
factors must be considered: reliability and validity. While psychometricians still argue about the relative 
importance of each of these concepts and what constitutes "good" reliability and validity, some general 
explanatory statements can help to clarify these test qualities. 

Reliability can be seen as the stability or consistency of the assessment. For instance, two assessments 
of the same student, done at the same time, should show similar results; two reviews of a teachers 
qualifications should result in similar conclusions. An instrument must be reliable if it is to be used to 
make decisions about whole-school program implementation or about students participating in the whole- 
school bilingual education program. Moreover, a measure that is not reliable cannot be valid. 

Validity -is more difficult to describe, in part because psychometricians arc changing their own views of 
validity (Angoff 1 988). As described by Mcssick (1988), validity asks whether the interpretation, uses, and 
actions based on assessment results arc appropriate. The Joint Committee of the American Educational 
Research Association, American Psychological Association, and National Council on Measurement adds 
that "validity... refers to the appropriateness, mean ingfuln ess, and usefulness of the specific inferences made 
from test scores' or assessment results (1985, p. 9). 

An instrument is reliable and valid only when it is used in the manner for which it was developed and 
for the purpose for which it was designed (including, of course, the students for whom it was designed). 
For example, a standardized, norm-referenced language arts test should not be used to determine students' 
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English proficiency. And, just as assessments must be reliable and valid, so must evaluations if they arc to be 
useful in the decision-making process. 

Assessment and Evaluation Plan Summary 

When considering the purpose of an assessment, whether it is to be used as part of a needs assessment, 
to measure progress or outcomes, or as part of an evaluation, there is one key element— if there is no real 
purpose for a particular assessment, do not do it! If the aim is to show change over time (e.g., change in 
teacher attitudes, improvement in the curriculum for LM students, or student growth in reading), then the 
assessment should be something that can be used several times throughout the year. If the purpose is to 
show that objectives for a program have been met, then the assessment should match those objectives. 

It also should be remembered that evaluation and assessment are tied together closely. Individual 
assessments for the various key indicators for school context, program implementation, and student 
outcomes arc collected to show progress. These assessments then are aggregated for the purpose of 
evaluation to show that a program is meeting its stated purposes. Program evaluation and assessment 
include accountability and ongoing modification to ensure a program's continued success. Regardless of 
their purpose, assessment and evaluation must take into account reliability and validity and must focus on 
making a determination as to whether or not the linguistic, cultural, and education needs of individual 
students and the community (i.e., employers, families, and groups of students) are met. 

IMPLICATIONS OF ASSESSMENT 

What implications do these assessment and evaluation approaches have for whole-school bilingual 
education programs? hive major issues, at this juncture, appear to be of most importance. 

1 . Time. As mentioned earlier, sufficient time will be needed to conduct the necessary needs assessment 
for each component of the evaluation as well as to field-test the validity and practicality of their 
assessment strategics. Equally important is the time needed to carry out the collaborative efforts with 
other funding agencies that have not been required before (e.g, coordination among Title I, Title VII, 
and Migrant Education Programs). 

2. Fundim . In order to carry out a process-oriented evaluation like the one proposed, more funding will 
be required than has been allocated in the past for evaluations. Support for initial and ongoing 
planning, continuous monitoring, new assessment strategies, summarizing, and report writing will be 
required if a meaningful and useful evaluation is to be conducted. 

3. Support . This is a key issue for success. Pvlost schools have not had to assess the implementation of 
their programs. Support in the form of technical assistance, for example, must be available to help 
school personnel collect the relevant information, report the necessary data, and account for their use 
of federal dollars appropriately. 

4. School expertise . In the spirit of collaboration and team-effort, staff, parents, students, and other key 
stakeholders must be trained and become experts on general assessment issues in order to participate in 
many aspects of the evaluation. School staff have the advantages of knowing the system that is in place, 
and being aware of the strengths and weaknesses of the day-to-day operations of the school. However, 
caution must be taken to allow evaluation teams the time to carry-out their responsibilities and develop 
the expertise required for reliable and valid assessments. 
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5. Assessment . Rather than testing different groups of students with different assessments or with the 
same, possibly inappropriate assessment, the whole-school bilingual program should look at ways to 
assess all studems comparably. Portfolio assessments are one way to accomplish this and should be 
considered by whole-school programs. The specific contents and scoring procedures might change 
depending upon the students grade level, language proficiency, and achievement, but the general 
portfolio contents would include the same types of materials. In this way, achievement levels can be 
shown to be comparable across classrooms within the whole-school program. 

6. Evaluators . Who will design and implement the evaluation? Who will conduct the observations and 
interviews, or lead the focus group(s)? The answers to questions such as these are based on (I) the 
expertise of the school administrators and staff, (2) the purpose of the evaluation, (3) the monies 
available, and (4) the time available. School staff must be allowed the opportunity to consider utilizing 
internal evaluators, external evaluators, or a combination of both. A determination of the best 
alternative will be based on the expertise, isolation, and complexity of the program. 

7. Evaluation. Most current evaluations deal almost solely with student outcomes. Within a whole- 
school bilingual education program, evaluation will also need to include the key indicators for program 
context and implementation. To ensure appropriate planning and evaluation, school data must be 
disaggregated by specific home language groups, language proficiency, and gender. 



CONCLUSIONS 

A whole-school bilingual education program is most likely to succeed when all its components 

context, implementation, and outcomes— are working in concert. In other words, when the context is 
positive for all those involved in the education of students (including parents, administrators, teaching 
staff, and students), then ihc implementation will be positive. The curriculum will be innovative and 
sensitive to the needs of all students in the school; staff will be empowered by training and decision-making 
opportunities; administrators will understand the needs of their students and staff and encourage collabo- 
rations between them and with the community-at-Iarge; families will be comfortable within the school 
environment; and learning strategies will be appropriate for the needs of LM and LEP students for whom 
teaching staff have high expectations. In being successful and seeing themselves as contributing and 
valuable members of the education community, all students, including language minority students, will 
develop improved self-concepts and will be motivated to work toward even higher standards. 
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